Generalized softmax networks for non-linear component extraction

Abstract. We develop a probabilistic interpretation of non-linear component extraction in neural networks that activate their hidden units according to a softmaxlike mechanism. On the basis of a generative model that combines hidden causes using the max-function, we show how the extraction of input components in such networks can be interpreted as maximum likelihood parameter optimization. A simple and neurally plausible Hebbian Δ-rule is derived. For approximatelyoptimal learning, the activity of the hidden neural units is described by a generalized softmax function and the classical softmax is recovered for very sparse input. We use the bars benchmark test to numerically verify our analytical results and to show competitiveness of the derived learning algorithms.

Gamma oscillations and object processing in the infant brain

An enduring controversy in neuroscience concerns how the brain “binds” together separately coded stimulus features to form unitary representations of objects. Recent evidence has indicated a close link between this binding process and 40-hertz (gamma-band) oscillations generated by localized neural circuits. In a separate line of research, the ability of young infants to perceive objects as unitary and bounded has become a central focus for debates about the mechanisms of perceptual development. Here we demonstrate that binding-related 40-hertz oscillations are evident in the infant brain around 8 months of age, which is the same age at which behavioral and event-related potential evidence indicates the onset of perceptual binding of spatially separated static visual features